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Motivation 
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• Is rainbird water throw going to wet the vehicle? 

• Answer it by smoothed particle hydrodynamics (SPH) modeling 



SSS Flow Test 39A, May 2004 



VOF Simulations 
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• 2 simulations using a 2-D structured mesh of rainbird nozzle mounted 
12' above the deck based on OpenFOAM multiphase flow solver. 

• Simulation 1 - "corner rainbird" case: Water injection at 112,500 gpm. 

• Simulation 2 - "center rainbird" case: Water injection at 55,250 gpm. 

• Both simulations were run up to 5 seconds. 
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Recommendation 
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• 3-D VOF 

• Smoothed Particle Hydrodynamics 
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SPH Formulation 
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SPH is a meshfree method with nodal collocation, spatial discretization, and kernel 
approximation. 

Starting with the conservation equation of mass and momentum: 


Dp 

Dt 


= —pV • v 


— = --VP + uV 2 v + g 
Dt p 


written in compact matrix form: A(f(r)) = Va + F, VreQ 

B(f{r)) = f, Vrer 

• Let f h (r) is an approximation of /(r): 

fir ) « f h (r ) = ^ Ni(r)fi 

i= l 

where fcrfir) is nodal value of f(r) at specified particle r t . 

N-(r) is the shape function used to interpolate field f(r) from f t 
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• For any test function v in the domain Q and boundary T, 


I v T A(f(r))df] + f v T B(f(r))dr = 0 
Jn Jr 


Test function v can be constructed by some basis function Oj 

and v 


v 


= y fojOj and v = y ftjOj 


1 = 1 


1 = 1 


leading to the final weighted residual function 


[ <S> T A(f h (r))d.n + [ <P T B(f h (r))dr = 0 

Jn Jr 


SPH Formulation 
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• Point collocation discretized the weighted residual function based on Dirac delta function 


• Dirac delta function has some useful properties: 

00 

^S(r)dr = 1 ^S(r-r')f(r')dr'= f(r) 

Q —oo 

• For a boundary value problem, 

A(f(r)) = 0, VreQ 
B(f(r)) = 0, Vrer 

• Use the delta function S(r f r) as test function, we can derive a set of collocation eqs: 

5(/‘(o))= 0, j = 1,2, ...,r 2 

where r 1 and r 2 are particles in Q and r, respectively 
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SPH Formulation 
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• In a Kernel approximation, the S function can be replaced by a smoothing 
function w(r-r’,h), which is an even function and satisfies the following 
conditions: 


J w(r-r', h)dr'= 1 


lim w(r-r\h) = S{r-r ') 

0 


w(r — r',h) = 0 when 


r-r 


> kh 


a 


where k defines the compact support of the smoothing function, and f(r) can be 

approximated as , r ,, , w ,, w , 

/ 0') = J /(r - )w(r- - r , h)dr 


a 


The integral form can be discretized by particle approximation: 

f\x) = f j w i (r^VJ i =Y,N l (r)f i 

i — 1 i-\ 

where w i (r)=w(r-r i ), and AV t is the volume of particle r t 
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SPH Formulation 
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In SPH, finite volume of particle is related to mass of particle through density 

m,=PAV t 

The approximate function can be written as 


/* 00 = X w i ( r ) A V ifi = X w < 00 — ft 

i = 1 i = 1 Pi 

The approximate solution of particle i is 


m , 


M Pj 

where w i} = wfa-ruh) , thus the density of particle i becomes: 


Support domain of 


smoothing function W 

Compact support 

• . 
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•• MO r,*\ • 

j 1 • \ y 

* » \ )•• 
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• • 


- * > • 
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W|rrrj,/i) 


Pi 


-I 


w tj mj 


1 = i 


The above equation shows particle density is based on smoothing the 
surrounding particle masses, therefore the name "smoothed particle". 
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Floating 
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Press 


1 .82e+004- 



I 

-5.87e+003 * 


14 



KSC ENGINEERING 
AND TECHNOLOGY 


Pump 


Vorticity Magnitude 


98.4 


0.00173 



ME. 
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Multi-GPU 

SPH 


GPUs: 64 x M2090 (BSC) 

MPI: Dynamic balancing 
Algorithm: Verlet & Wendland 
Particles: 1,015 Millions 
Steps: 237,342 
Runtime: 91.9 hours 
Physical time: 12 seconds 



Time: 0.00 s 



Computational Resource 



• Current Beast: 

• Dual Quadro 6000, 6 GB, 448 CUDA GPU 

• 256 GB RAM 

• Dual Intel Xeon E5-2690 

• 512 GB SSD, 3 TB SATA (Win7) 

• 256 GB SSD, 2 TB SATA (Debian Linux) 

• Upgrade Beast: 

• Tesla K40 (12 GB GDDR5, 2880 CUDA cores) for computations (4.29 Tflops) 

• Quadro K6000 (12 GB GDDR5, 2880 CUDA cores) for graphic rendering 
(2560x1600) 

• 1 TB SSD Drives 
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Approach 
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• Import full ML CAD Model 

• Run multiple rainbirds with variable flowrates and timing sequence 

• Activate vehicle motion with velocity/acceleration profile extracted 
from MSFC trajectory analysis 
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Water Tank & Rainbird 
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Rainbird 
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SPH Rainbird 
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Time: 0.000000 sec 
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SPH Rainbird 
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Time: 5.000000 sec Vel (m/s) 
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Verification 
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• Traj Plots CSE with bypass (from Nick Moss' Rainbird Water Throws) 

• North Corner Rainbirds: 28,381 GPM 

• South Rainbirds: 56,762 GPM 


North Corner Rainbirds 



♦ T-2.5 ■T-3.0 AT- 1.5 XT-1 JCT-0 5 *T-0 +T+0-S -T+l — T+1S AT + 2 BT+2.5 


South Rainbirds 



♦ T-2.5 BT-2.0 AT-1.5 XT-1 XT-0.5 #T-0 +T+0.5 -T+l -T+1.5 AT+2 BT+2.5 
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Verification 
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• Traj Plots CSE with bypass (from Nick Moss' Rainbird Water Throws) 

• North Corner Rainbirds: 6.01m - 7.433m 

• South Rainbirds: 7.0m - 8.7m 


North Corner Rainbirds 



South Rainbirds 



♦ T-2.5 ■T-7.0 AT-1.5 XT-1 XT-0.5 +T+0.5 -T+l -T+1.5 ♦ T+? bT+2.5 
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Verification 
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Verification 
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Verification 
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Verification 
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Verification 
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Verification 
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Verification 
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Flow time=3.5s 
Total time=6.5s 


44,084 GPM 


Time: 0 sec 


69,982 GPM 


51,786 GPM 
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Verification 
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Water volume flow was 
based on a maximum 
nominal rainbird flow 
during T-10 to T+20sec 
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Next Iteration 
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• Correct rainbird flow timing and volume flow rates; make it variable based 
on Nominal or Abort operation to reduce conservatism. 

• Correct vehicle motion; add correct velocity or acceleration profile 

• Add geometry complexity to include TSM, ML deck roughness, and exhaust 
hole features 
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Nominal RB Flows and SLS Motion 
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Time 

Nominal Lai 

North East 

North Center 

North West 

South East 

South West 

-11 

150.40 

23.69 

27.82 

23.69 

37.60 

37.60 

-10.9919 

150.40 

23.69 

27.82 

23.69 

37.60 

37.60 

-10.9838 

150.40 

23.69 

27.82 

23.69 

37.60 

37.60 

-10.9757 

150.40 

23.69 

27.82 

23.69 

37.60 

37.60 

-10.9677 

150.40 

23.69 

27.82 

23.69 

37.60 

37.60 

-10.9596 

150.40 

23.69 

27.82 

23.69 

37.60 

37.60 

-10.9515 

150.40 

23.69 

27.82 

23.69 

37.60 

37.60 

-8.18E-04 

223970.34 

35275.33 

41434.51 

35275.33 

55992.59 

55992.59 

7.27E-03 

223906.23 

35265.23 

41422.65 

35265.23 

55976.56 

55976.56 

9.62E-02 

224876.86 

35418.11 

41602.22 

35418.11 

56219.22 

56219.22 

0.104321 

224768.70 

35401.07 

41582.21 

35401.07 

56192.18 

56192.18 

0.201373 

230832.24 

36356.08 

42703.96 

36356.08 

57708.06 

57708.06 

0.306512 

251825.97 

39662.59 

46587.80 

39662.59 

62956.49 

62956.49 

0.403564 

256285.29 

40364.93 

47412.78 

40364.93 

64071.32 

64071.32 

0.500615 

257607.13 

40573.12 

47657.32 

40573.12 

64401.78 

64401.78 

0.605754 

260813.08 

41078.06 

48250.42 

41078.06 

65203.27 

65203.27 

0.702806 

280546.00 

44186.00 

51901.01 

44186.00 

70136.50 

70136.50 

0.807945 

286032.51 

45050.12 

52916.01 

45050.12 

71508.13 

71508.13 

0.904997 

288221.27 

45394.85 

53320.93 

45394.85 

72055.32 

72055.32 

1.002048 

290497.30 

45753.32 

53742.00 

45753.32 

72624.33 

72624.33 








19.91094 

193616.32 

30494.57 

35819.02 

30494.57 

48404.08 

48404.08 

19.91902 

193195.61 

30428.31 

35741.19 

30428.31 

48298.90 

48298.90 

19.92711 

192774.99 

30362.06 

35663.37 

30362.06 

48193.75 

48193.75 

19.9352 

192355.81 

30296.04 

35585.82 

30296.04 

48088.95 

48088.95 

19.94329 

191938.81 

30230.36 

35508.68 

30230.36 

47984.70 

47984.70 

19.95137 

191520.57 

30164.49 

35431.31 

30164.49 

47880.14 

47880.14 

19.95946 

191100.31 

30098.30 

35353.56 

30098.30 

47775.08 

47775.08 

19.96755 

190684.90 

30032.87 

35276.71 

30032.87 

47671.23 

47671.23 

19.97564 

190264.95 

29966.73 

35199.02 

29966.73 

47566.24 

47566.24 

19.98372 

189846.94 

29900.89 

35121.68 

29900.89 

47461.74 

47461.74 

19.99181 

189427.82 

29834.88 

35044.15 

29834.88 

47356.96 

47356.96 

19.9999 

189005.44 

29768.36 

34966.01 

29768.36 

47251.36 

47251.36 


Ascent Elevation 

TPS 

time 

elev 

elev 

sec 

ft 

ft 

0 

0 

5.098333 

0.1 

0 

5.098333 

0.2 

0 

5.098333 

0.3 

0.12501 

5.223343 

0.4 

0.333347 

5.43168 

0.5 

0.625018 

5.723351 

0.6 

1.000021 

6.098354 

0.7 

1.458358 

6.556691 

0.8 

2.000029 

7.098362 

0.9 

2.625033 

7.723366 

1 

3.33337 

8.431703 

1.1 

4.125041 

9.223374 

1.2 

5.000045 

10.09838 

1.3 

6.39535 

11.49368 

1.4 

7.8414 

12.93973 

1.5 

9.42375 

14.52208 

1.6 

11.1424 

16.24073 

1.7 

12.99735 

18.09568 

1.8 

14.9886 

20.08693 

1.9 

17.11615 

22.21448 

2 

19.38 

24.47833 

2.1 

21.78015 

26.87848 

2.2 

24.3166 

29.41493 

2.3 

26.98935 

32.08768 

2.4 

29.7984 

34.89673 

2.5 

32.74375 

37.84208 

2.6 

35.8254 

40.92373 

2.7 

39.04335 

44.14168 

2.8 

42.3976 

47.49593 

2.9 

45.88815 

50.98648 

3 

49.515 

54.61333 
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Nominal RB Flows and SLS Motion 
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Time (sec) 



vel (m/s) 






NE 

NC 

NW 

SE/SW 

NE/NW 

NC 

SE/SW 

-8.18E-04 

0.2368 

0.2781 

0.2368 

0.3759 




7.27E-03 

0.2367 

0.2781 

0.2367 

0.3758 




9.62E-02 

0.2377 

0.2793 

0.2377 

0.3774 




0.1043212 

0.2376 

0.2791 

0.2376 

0.3772 




0.2013728 

0.2440 

0.2867 

0.2440 

0.3874 




0.306512 

0.2662 

0.3127 

0.2662 

0.4226 




0.4035636 

0.2710 

0.3183 

0.2710 

0.4301 




0.5006152 

0.2724 

0.3199 

0.2724 

0.4323 




0.6057544 

0.2757 

0.3239 

0.2757 

0.4377 




0.7028061 

0.2966 

0.3484 

0.2966 

0.4708 




0.8079453 

0.3024 

0.3552 

0.3024 

0.4800 




0.9049969 

0.3047 

0.3579 

0.3047 

0.4837 

0.2677 

0.3145 

0.4250 

1.002048 

0.3071 

0.3608 

0.3071 

0.4875 




1.107188 

0.3207 

0.3768 

0.3207 

0.5091 




1.204239 

0.3302 

0.3879 

0.3302 

0.5241 




1.301291 

0.3327 

0.3908 

0.3327 

0.5281 




1.40643 

0.3360 

0.3947 

0.3360 

0.5334 




1.503482 

0.3433 

0.4032 

0.3433 

0.5449 




1.600533 

0.3533 

0.4150 

0.3533 

0.5608 




1.705673 

0.3596 

0.4224 

0.3596 

0.5708 




1.802724 

0.3594 

0.4222 

0.3594 

0.5705 




1.907863 

0.3654 

0.4292 

0.3654 

0.5800 

0.3408 

0.4003 

0.5409 

2.004915 

0.3737 

0.4389 

0.3737 

0.5931 




2.101967 

0.3796 

0.4459 

0.3796 

0.6026 




2.207106 

0.3803 

0.4467 

0.3803 

0.6036 




2.304157 

0.3848 

0.4520 

0.3848 

0.6108 




2.401209 

0.3910 

0.4593 

0.3910 

0.6207 




2.506348 

0.3966 

0.4659 

0.3966 

0.6296 





Time (sec) 



vel (m/s) 






NE 

NC 

NW 

SE/SW 

NE/NW 

NC 

SE/SW 

2.6034 

0.4006 

0.4706 

0.4006 

0.6359 




2.700452 

0.4015 

0.4716 

0.4015 

0.6374 




2.805591 

0.4069 

0.4780 

0.4069 

0.6459 




2.902642 

0.4111 

0.4828 

0.4111 

0.6525 

0.3926 

0.4612 

0.6232 

3.007782 

0.4165 

0.4892 

0.4165 

0.6611 




3.104833 

0.4167 

0.4894 

0.4167 

0.6614 




3.201885 

0.4204 

0.4938 

0.4204 

0.6673 




3.307024 

0.4242 

0.4983 

0.4242 

0.6734 




3.404076 

0.4289 

0.5038 

0.4289 

0.6808 




3.501127 

0.4301 

0.5052 

0.4301 

0.6827 




3.606266 

0.4324 

0.5079 

0.4324 

0.6863 




3.703318 

0.4355 

0.5115 

0.4355 

0.6913 




3.80037 

0.4384 

0.5150 

0.4384 

0.6959 




3.905509 

0.4416 

0.5188 

0.4416 

0.7010 

0.4285 

0.5033 

0.6801 

4.002561 

0.4427 

0.5200 

0.4427 

0.7027 




4.1077 

0.4449 

0.5226 

0.4449 

0.7062 




4.204751 

0.4477 

0.5259 

0.4477 

0.7107 




4.301803 

0.4508 

0.5295 

0.4508 

0.7155 




4.406942 

0.4517 

0.5306 

0.4517 

0.7170 




4.503994 

0.4529 

0.5320 

0.4529 

0.7189 




4.601046 

0.4554 

0.5349 

0.4554 

0.7228 




4.706185 

0.4580 

0.5379 

0.4580 

0.7269 




4.803236 

0.4593 

0.5395 

0.4593 

0.7291 




4.900288 

0.4602 

0.5406 

0.4602 

0.7305 

0.4524 

0.5313 

0.7180 

5.005427 

0.4617 

0.5423 

0.4617 

0.7328 





RB Flows Based on 3.5-m Water Tank 


time 

elev 

elev 

vel 

accel 


sec 

ft 

m 

m/s 

m/s2 


0 

0 

0 




0.1 

0 

0 

0 



0.2 

0 

0 

0 

0 


0.3 

0.12501 

0.038103 

0.38103 

r 1.905152 


0.4 

0.333347 

0.101604 

0.635011 

2.539807 


0.5 

0.625018 

0.190505 

0.889013 

2.54002 


0.6 

1.000021 

0.304806 

1.143009 

2.539959 


0.7 

1.458358 

0.444508 

1.397011 

2.54002 


0.8 

2.000029 

0.609609 

1.651013 

2.54002 


0.9 

2.625033 

0.80011 

1.905012 

2.53999 


1 

3.33337 

1.016011 

2.159011 

2.53999 


1.1 

4.125041 

1.257312 

2.413013 

2.54002 


1.2 

5.000045 

1.524014 

2.667012 

2.53999 

2.53998 

1.3 

6.39535 

1.949303 

4.25289 

15.85877 


1.4 

7.8414 

2.390059 

4.40756 

1.546708 


1.5 

9.42375 

2.872359 

4.823003 

4.154424 


1.6 

11.1424 

3.396204 

5.238445 

4.154424 


1.7 

12.99735 

3.961592 

5.653888 

4.154424 


1.8 

14.9886 

4.568525 

6.06933 

4.154424 


1.9 

17.11615 

5.217003 

6.484772 

4.154424 


2 

19.38 

5.907024 

6.900215 

4.154424 


2.1 

21.78015 

6.63859 

7.315657 

4.154424 


2.2 

24.3166 

7.4117 

7.7311 

4.154424 


2.3 

26.98935 

8.226354 

8.146542 

4.154424 


2.4 

29.7984 

9.082552 

8.561984 

4.154424 


2.5 

32.74375 

9.980295 

8.977427 

4.154424 


2.6 

35.8254 

10.91958 

9.392869 

4.154424 


2.7 

39.04335 

11.90041 

9.808312 

4.154424 


2.8 

42.3976 

12.92279 

10.22375 

4.154424 


2.9 

45.88815 

13.98671 

10.6392 

4.154424 


3 

49.515 

15.09217 

11.05464 

4.154424 
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V (m/s) 
2 5 5 
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Full Simulations 
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ML Geometry 
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Correct Flow Ramp-up 
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South Rainbird Peak F|0W = 56 - 762GPM 

Time: -5 sec 
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Correct Flow Ramp-up 


South Rainbird 
Time: -5 sec 


Peak Flow = 56,762GPM 



V (m/s) 

2 4 

0 6 
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Double Jet 
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Source: "Hydraulic Model Studies of Spray Nozzles for a Sound Suppression Water 
System," Joseph Wetzel and John Ripken, University of Minnesota, March 1977. 
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Spray Patterns 
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Source: "Hydraulic Model Studies of Spray Nozzles for a Sound Suppression Water 
System," Joseph Wetzel and John Ripken, University of Minnesota, March 1977. 


Type 1, 50,000 GPM 

Nozzle span angle = 100°, Jet fan angle = 80° 


(Not Shown) 

Type 2, 40,000 GPM 

Nozzle span angle = 190°, Jet fan angle = 150° 


Nozzle span angle = 190° 
Jet fan angle « 120° 
66,543 GPM 


Nozzle span angle = 180° 
Jet fan angle « 80° 
56,652 GPM 


• 1:2.8 scale ratio 

• Dissimilar pipe transition 
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Jet Spray Patterns 
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Time: -5 sec 

Vel (m/s) 



oi 


No SLS (-5s to 6.6s) 
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Time: -5 sec 


Vel (m/s) 

12 - 


With SLS (-5s to 9s) 
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Abort Simulation 
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Simulation window 


Individual Rainbifd Flows (ABORT) 

65 000 



NW- PIPE 240 NC- PIPE 239 NE - PIPE 39 SW- PIPE 237 SE- PIPE 238 
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Abort Simulation 
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Simulation window 


TOTAL POST LIFTQFF FLOW (ABORT) 

250000 
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Region of Interest 
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Geometry Issues 
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Water Depth 
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Water Depth 
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Water Depth 



Time: 5 
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Water Depth 
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Water Depth 
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Summary 



KSC ENGINEERING 
AND TECHNOLOGY 


• New GPU cards were installed and performing as expected 

• Cameras will get minimal impact 

• Water puddle is as deep as 0.3m = 12" 

• TSM gap could result in shallow water depth 
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Updates 
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• Quadra K600 outperformed Tesla K40c 

• FixTSM gap 

• Incorporate design of water barrier for HBOI 

• Install camera locations 
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Abort Simulation (fixed TSM) 
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Abort Simulation (fixed TSM) 
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NoTSM Gap 
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• Water puddle as deep as 0.4m = 16" near the TSM and on the South side 


72 





KSC ENGINEERING 
AND TECHNOLOGY 



Fixed TSM 
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Fixed TSM 



74 





KSC ENGINEERING 
AND TECHNOLOGY 


Fixed TSM 
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Fixed TSM 
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ml 
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Fixed TSM 
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Forward Plan 
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• Build a multi-GPU cluster and equip the Beast with the best resources 

• Recruit doctoral student and post doc through Graduate STEM 
Fellowship to conduct research in meshfree method 

• Collaborate with UCF (A. Kassab), University of Cincinnati (G.R. Liu) 
and University of Manchester Research Group (A. Crespo) 
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• Free open-source SPHysics code: 
http://wiki.manchester.ac.uk/sphvsic 

• GPU-SPHysics: a GPU-based SPH model for free-surface flows 
http://www.ce.ihu.edu/dalrymple/GPU 

• SPHERIC = SPH European Research Interest Community: 
http://wiki.manchester.ac.uk/spheric 
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